Application of artificial intelligence technology in spectral analysis

Typical spectral analysis workflow usually includes data preprocessing, feature selection and multivariate modeling process. Principal component analysis (PCA), partial least squares (PLS), artificial neural networks (ANN) and support vector machines (SVM) are commonly used multivariate modeling methods for qualitative and quantitative spectral analysis.

In spectral analysis, preprocessing and feature selection have a great impact on the performance of multivariable models. Various preprocessing methods have been developed to eliminate the artifacts of spectral signals, which are usually derived from the acquisition environment and human operations. Different combinations and sequences of preprocessing methods lead to different modeling performance. Misuse of preprocessing will distort the original signal and reduce the accuracy of spectrum analysis. Feature selection usually results in partial information loss, which is unnecessary for obtaining information from data. In general, although a given preprocessing and feature selection process is recommended for a particular dataset, it may not work well for other analysis tasks due to the different sources of artifacts. Therefore, an integrated data-driven analysis approach is needed to remove artifacts and extract relevant information directly from spectral data, thereby reducing the need for prior knowledge of datasets.

Recent advances in artificial intelligence have shown that data-driven deep learning techniques can discover complex structures in large data sets and extract key features from the data without manually designing features. Deep learning models are multilayer stacks of nonlinear modules that learn patterns from raw inputs and gradually transform them into higher-level representations. With enough of these transformations, you can learn highly complex functions. The success of deep learning is the result of increasing data volume and computing power. Some popular deep learning architectures have found satisfactory applications in 2D and 3D data analysis. For example, deep convolutional neural networks have made breakthroughs in the field of computer vision, while recurrent neural networks have shown better performance in the analysis of sequence data such as text and speech. Deep learning has been successfully applied in many industrial fields, including medicine, bioinformatics, remote sensing, and transportation.

Recent studies have shown that deep neural networks can be applied to vibration spectroscopy. Since 2017, there have been more and more researches on deep learning methods, convolutional neural networks and self-coding networks for 1D vibration spectral data processing. The advanced architecture of deep learning and embedded regularization techniques significantly reduce the risk of overfitting, thus solving the main problem of traditional neural networks. Compared with other stoichiometric methods, deep neural networks can learn critical patterns hierarchically from the original spectrum, which reduces human factors in preprocessing and feature selection, and improves the accuracy and robustness of the model.